feat(studio): share native audio and upscale commands by IAnMove · Pull Request #280 · IAnMove/hocuspocus

IAnMove · 2026-09-09T15:17:20Z

What changes

Studio speech, music, SFX and Tools upscale use the same durable operations as external MCP clients. Wizard displays the exact command before admission; recovery retains its intention, receipt and native task. A deliberate new intention creates another job.

Music preserves literal lyrics, caption and description. ACE-Step accepts an omitted caption; MiniMax Music 3 requires it before admission. SFX supports text-only MMAudio and canonical video guides. Stale video-model options no longer alter SFX durations. Admission validates installed dependencies and resource identity; SFX rechecks them before model work.

SFX packs use distinct, stable per-clip intentions and preserve every admitted receipt, including partial failures or failed navigation. Explicit authored JSON clips and negative prompts survive missing or rewritten LLM actions; an incomplete pack cannot fall through to video generation. The server recovers each admitted clip; browser-independent pack scheduling remains P9 work.

This draft incorporates development d88c85d5, rechecked by fetch. Published candidate: b18b4441. PRs #284, #285, #286, #287 and #288 were integrated locally and hardened in this branch. The draft has not been merged into development, and main has not been published.

Audio references are isolated by sub-tab and restored with their displayed filenames. Loading saved settings retains the previous tab's references. Speech voice counts cannot discard newly selected Music references, rename voices from lyrics or overwrite Music selectors. Entering Audio selects the tab matching the restored model; late defaults cannot disable a restored reference. A selected voice file is visible and removable without deleting its speaker.

Load Settings restores SFX descriptions, negatives, zero-valued controls and recorded audio duration. An empty SFX box cannot submit leftover Speech/Music text; direct MCP prompt-only commands remain valid. Gallery restore/reroll capture the clicked filename and workspace; newer requests and workspace changes invalidate old responses. Generate shows preparation rather than optimistic queue acceptance, guards rapid duplicate clicks, and excludes failed placeholders from its active count.

SFX video guides now leave the active form when switching to Speech/Music and return with SFX. Its description, negative prompt and strength cannot block either other tab. A real browser pass also exposed a Music → Speech failure on hidden alt_guidance_scale; the form projection now removes that music-only field and lyrics_language. Direct command envelopes retain their closed validation. Audio admission guards share one snapshot definition with tab-specific controls.

Validation

Required CI and both Cursor checks on b18b4441 PASS (run34405517373): 3,022 Python passed, 9 skipped; UI checks, Chromium boot and Windows Speech export E2E pass. Independent QA is neutral, not independent approval. Luna workers exhausted their quota; independent review of new changes remains pending.
Local b18b4441: 1,441 UI tests pass, ESLint, TypeScript, i18n, build and code-health ratchet against development pass. Entry JS gzip: 324,680/327,680 B. Production LOC +8,223 against a budget of 8,229; no baseline or budget increases. A previous candidate exceeded that LOC budget; repeated typed-submit error blocks were consolidated and unreachable button branches removed, with tests retained.
New directed tests reproduce SFX form residue, restore literals/durations including zero controls, rapid repeat clicks, delayed/out-of-order metadata responses and a workspace change during reroll. Existing Viggle restore tests pass. Browser03 on a3e377 passed restore/rejection; browser04 on094 reproduced the wrong-file gallery selection and is retained as FAIL. Browser05 on07 timed out loading Vite before entering the app; browser06 on the same07 PASS: empty-SFX rejection with 0 POST, both exact saved sounds/durations restored, and button labels Generate → Preparing… → Generate. The servers stayed online; no product change was made between05/06. No new inference has been performed for these form fixes.
Browser guide round trip on b18b4441: PASS, one real video upload and SFX → Music → Speech → SFX. Three typed requests preserve authored text, omit foreign controls and restore the original SFX guide. Requests were intercepted with an explicit 422 before server admission: zero new jobs/inferences. The first pass on3c9b failed on the hidden Music control and remains recorded as FAIL; the corrected pass is separate. Public screenshots explain the deliberate rejection messages.
Real browser on 5cb8107c: one file upload, Speech → Music → SFX → Speech restores the exact voice/file/selector. No generation was performed for this form test. Earlier browser failures exposed a wrong initial tab/model and a late-defaults selector reset; harness navigation/upload mistakes are recorded separately. Screenshots and results are published.
Real pack on 94ab0068: one Wizard request, two V2 admissions, distinct stable child intentions and two native MMAudio WAVs, each 1.022 s / 44.1 kHz mono. Native processing: 38.35 s and 126.16 s. Both official MCP replays retain their original receipts and one task per intention; zero active jobs afterward. Full decode, audible Chromium playback and UI download pass, with matching native/MCP/UI/LAN hashes.
The pack observer timed out downloading the second output after both jobs had completed. Its original FAIL is preserved; subsequent recovery is recorded separately and does not generate again. Two prior attempts were blocked before native admission and exposed the fixed parser defects. The trace records one Wizard request and two LLM endpoint POSTs; these are not counted as one provider call.
Earlier real SFX: text WAV3.02s on74d, guided MP4 on0220 preserving all54 original frames and replacing audio. Both validated through Wizard, official MCP, full decode, browser playback and identical downloads. Earlier Music on60019: ACE-Step XL,20s stereo48kHz WAV; original observer timeout preserved, GET-only recovery and MCP replay pass. Earlier Tools Lanczos1024², Kugel speech3.6s and Image#275 evidence retained. Historical inference is not attributed to a later commit.

Remaining work

P5 still has remaining Studio/Tools modalities. P6–P10 are unfinished, including browser-independent editorial workflows and Video3D export. Reference drafts persist for the browser session; full prompt/caption/duration stashing and cross-reload drafts remain outside this change. SFX can still display a 20 s slider while retaining an invalid larger duration (the command rejects it); a restored guide may display a legacy path until submission canonicalizes it. PR283's count-based deletion heuristic was not integrated because it can erase a newly selected Music reference. H3 processor reference arrays retain a native scalar-settings limitation.

Evidence: outputs/wizard-mcp-execution-20260909/. Samples and screenshots: http://192.168.1.87:4192/wizard-mcp-20260909/ . SFX pack: http://192.168.1.87:4192/wizard-mcp-20260909/sfx/#pack . Audio UI regression: http://192.168.1.87:4192/wizard-mcp-20260909/audio-reference-fixes/ .

PR281 navigation/styles are preserved. The only shared-file compatibility change in VideoInfoBar is wrapping two onClick callbacks so React events cannot be mistaken for an optional output source; its hunks are separate from PR281.

Current form-fix report and screenshots: http://192.168.1.87:4192/wizard-mcp-20260909/sfx-form-fixes/ . All five report files returnHTTP200 and match local hashes.

Audio guide report and screenshots: http://192.168.1.87:4192/wizard-mcp-20260909/audio-guide-fixes/ . Seven report files return HTTP200 with matching hashes.

Note

Medium Risk
Large change to admission, recovery, and queue dispatch across audio/tools paths; incorrect adapter or recovery matching could mis-route jobs or reject valid recoveries.

Overview
Extends the existing image generation command stack into a pluggable native generation facade so Studio speech, music, SFX, and Tools upscale share the same durable intent → receipt → task → FIFO path as MCP/Wizard clients.

ImageGenerationCommands now registers NativeGenerationOperation adapters (generation.speech, generation.music, generation.sfx, tools.upscale) with per-operation freeze/prepare/catalog, operation-aware replay fingerprints, and native_worker so recovery/dispatch only runs tool workers when provenance matches a canonical admission. HTTP/MCP discovery merges adapter catalogs; reference resolution accepts media_kind for audio/video.

Each modality adds a closed v2 spec (Pydantic freeze + fingerprint), preparation (model/install checks, no queue side effects), and resource inspection (speech/music audio refs, SFX video guides + MMAudio file sets). Generation input prep honors prepared_studio_speech / prepared_studio_audio for admitted audio params.

Runtime hooks: deferred prep consults native_worker first; legacy tools upscale can admit_generation_command as tools.upscale. SFX uses prepared_sfx_execution to re-verify receipt, params, and guide identity before inference—skipping legacy MMAudio downloads for typed jobs and failing closed on missing/changed guides instead of silent text-only fallback.

^{Reviewed by Cursor Bugbot for commit b18b444. Configure here.}

…red-audio-tool-commands # Conflicts: # app/services/image_generation_commands.py

…red-audio-tool-commands

# Conflicts: # app/services/image_generation_commands.py

…ol-commands

github-actions · 2026-09-09T15:17:34Z

PR Review — Loreframe Studio

Risk: medium
Scope: 146 file(s); +21953/-1116; React UI, backend services, docs, repo scripts / CI

Automated review from scripts/analyze_pr.py. This is a heuristic pass (no LLM) so humans still own the merge decision.

Findings

medium — Dangerous dynamic execution (tests/test_studio_sfx_native_worker.py)
tests/test_studio_sfx_native_worker.py adds exec(). Confirm the input is trusted and sandboxed.
medium — Dangerous dynamic execution (tests/test_tools_command_runtime.py)
tests/test_tools_command_runtime.py adds exec(). Confirm the input is trusted and sandboxed.
medium — Large pull request
21953 additions / 1116 deletions. Reviewers will have an easier time with smaller, focused PRs.
medium — Very large file change (app/services/studio_music_spec.py)
app/services/studio_music_spec.py adds 503 lines. Consider splitting the PR.
medium — Very large file change (app/services/studio_sfx_spec.py)
app/services/studio_sfx_spec.py adds 458 lines. Consider splitting the PR.
medium — Very large file change (app/services/studio_speech_preparation.py)
app/services/studio_speech_preparation.py adds 510 lines. Consider splitting the PR.
medium — Very large file change (app/services/studio_speech_spec.py)
app/services/studio_speech_spec.py adds 637 lines. Consider splitting the PR.
medium — Very large file change (app/services/tools_upscale_preparation.py)
app/services/tools_upscale_preparation.py adds 412 lines. Consider splitting the PR.
medium — Very large file change (tests/test_tools_upscale_preparation.py)
tests/test_tools_upscale_preparation.py adds 404 lines. Consider splitting the PR.
medium — Very large file change (ui/src/api/generationCommandClient.ts)
ui/src/api/generationCommandClient.ts adds 727 lines. Consider splitting the PR.
medium — Very large file change (ui/src/api/musicCommandCatalog.json)
ui/src/api/musicCommandCatalog.json adds 1395 lines. Consider splitting the PR.
medium — Very large file change (ui/src/api/sfxCommandCatalog.json)
ui/src/api/sfxCommandCatalog.json adds 535 lines. Consider splitting the PR.
medium — Very large file change (ui/src/api/speechCommandCatalog.json)
ui/src/api/speechCommandCatalog.json adds 1973 lines. Consider splitting the PR.
medium — Very large file change (ui/src/api/toolsCommandCatalog.json)
ui/src/api/toolsCommandCatalog.json adds 491 lines. Consider splitting the PR.
medium — Very large file change (ui/src/features/studio/musicGenerationSpec.ts)
ui/src/features/studio/musicGenerationSpec.ts adds 427 lines. Consider splitting the PR.
medium — Very large file change (ui/tests/musicGenerationCommands.test.ts)
ui/tests/musicGenerationCommands.test.ts adds 425 lines. Consider splitting the PR.
medium — Very large file change (ui/tests/studioMusicCommandPresentation.test.tsx)
ui/tests/studioMusicCommandPresentation.test.tsx adds 452 lines. Consider splitting the PR.
low — UI changed — rebuild before merge
Run cd ui && npm run build (CI already does this). Pinokio Update rebuilds for end users; keep ui/dist untracked.

Changed files

added: app/routers/studio_music_commands.py, app/routers/studio_sfx_commands.py, app/routers/studio_speech_commands.py, app/routers/tools_upscale_commands.py, app/services/native_generation_operation.py, app/services/studio_music_preparation.py, app/services/studio_music_spec.py, app/services/studio_sfx_commands.py, app/services/studio_sfx_execution.py, app/services/studio_sfx_preparation.py, app/services/studio_sfx_resources.py, app/services/studio_sfx_spec.py, app/services/studio_speech_preparation.py, app/services/studio_speech_resources.py, app/services/studio_speech_spec.py, app/services/tools_upscale_commands.py, app/services/tools_upscale_preparation.py, app/services/tools_upscale_spec.py, docs/development/MUSIC_COMMANDS.md, docs/development/SFX_COMMANDS.md
… and 76 more
modified: app/_launch_runtime.py, app/routers/image_generation_commands.py, app/services/image_generation_commands.py, app/services/image_generation_runtime.py, app/services/studio_image_resources.py, app/services/wangp_submission.py, docs/development/LABS_WIZARD_ACTION_MATRIX.md, tests/fixtures/architecture_wire_inventory.json, tests/fixtures/labs_wizard_action_matrix.json, tests/test_h3_preplan_job_contract.py, ui/e2e/helpers/apiRoutes.ts, ui/e2e/specs/tools-background-removal.spec.ts, ui/src/api/imageCommandCatalog.json, ui/src/api/imageGenerationCommands.ts, ui/src/components/MainContent/MediaFeedItem.tsx, ui/src/components/MainContent/VideoInfoBar.tsx, ui/src/components/Sidebar/AudioModeSection.tsx, ui/src/components/Sidebar/GenerateButton.tsx, ui/src/components/Sidebar/MusicControls.tsx, ui/src/components/Sidebar/SfxControls.tsx
… and 30 more

CONTRIBUTING checklist

python scripts/verify_clean_repo.py
python -m compileall -q app/services app/launch.py scripts
cd ui && npm run build if the UI changed
No weights, CivitAI sidecars, or generated guides
Stays local-first (no required accounts / telemetry)

Posted by the repo PR review workflow. Re-runs on each push to the PR.

github-actions · 2026-09-09T15:21:39Z

Code health

Quality score: 58.6/100

Higher is better. The score is a trend dashboard; the independent ratchet below remains the CI gate.

Component	Weight	Current	Change
Cyclomatic health	45%	56.1	+0.7
File concentration	25%	66.4	+2.2
Oversized-file debt	20%	42.3	+2.2
Modularity	10%	82.9	+1.9

Change vs PR base: +1.5 points.

Metric	Value
Production LOC	282,530
Production files	853
Test LOC	111,623
Functions measured	18,628
Functions complexity ≥ 15	865
Maximum complexity	674
Policy	code-health-policy-v1
HEAD	`c22e19d3188ff51b1147943789801a78dea34c57`
Base	`d88c85d543100dad3469014ff0d4c7bd7b242222`
UI measurement	complete

Markdown, JSON catalogs and tests are out of this table. Only app/ runtime + ui/src TS/JS count.

Most complex functions

Complexity	Where
674	`app/wgp.py:7221` `generate_video`
379	`ui/src/stores/useStore.ts:4118` `Async method 'startGeneration'`
355	`app/_launch_runtime.py:23653` `_run_generation`
308	`app/wgp.py:12363` `generate_video_tab`
264	`ui/src/stores/useStore.ts:8663` `Async method 'loadSettingsFromOutput'`
262	`ui/src/components/Sidebar/SceneAnimatorPanel.tsx:496` `Function 'SceneAnimatorPanel'`
258	`app/services/director/planners/short_film.py:3433` `ShortFilmPlanner._plan_story_driven`
248	`app/services/director_pipeline.py:13607` `_run_video_generation`
245	`app/services/director_pipeline.py:7712` `_run_pipeline`
233	`ui/src/features/agent/agentActions.ts:1152` `Function 'parseAction'`
226	`app/services/director_pipeline.py:6541` `update_comic_preview`
225	`ui/src/features/agent/agentActions.ts:2911` `Async function 'executeAgentActions'`

Trend vs baseline

Metric	Δ
Production LOC	+8,223
Test LOC	+7,224
Functions ≥ 15	+4
Maximum complexity	+0

Warnings

production LOC increased by +8,223
functions at complexity >= 15 increased by +4
complexity hotspot ui/src/components/MainContent/MediaFeedItem.tsx rose 122 -> 123
complexity hotspot ui/src/components/Sidebar/ToolsPanel.tsx rose 21 -> 24
complexity hotspot ui/src/features/agent/capabilityRunner.ts rose 23 -> 24
hotspot app/_launch_runtime.py increased by +36 lines
hotspot ui/src/stores/useStore.ts increased by +15 lines
hotspot ui/src/features/agent/capabilityRegistry.ts increased by +3 lines
hotspot ui/src/features/agent/applicationAdapters.ts increased by +20 lines

Ratchet passed.

# Conflicts: # ui/src/i18n/locales/en/studio.json # ui/src/i18n/locales/es/studio.json

…checks

Wizard queue_sfx_pack reused one capability commandId as every clip's generation.sfx intent_id. Distinct prompts then 409'd after the first clip; identical prompts silently replayed the first receipt. Co-authored-by: ignaciodelcano+dcl <ignaciodelcano+dcl@gmail.com>

Studio SFX owns MMAudio_prompt. Audio sub-tabs share one params map, so Speech or Music lyrics stayed in prompt. startGeneration used MMAudio_prompt ?? prompt, and the closed command accepted that leftover as the sound description. An empty SFX box then generated the previous script. Co-authored-by: ignaciodelcano+dcl <ignaciodelcano+dcl@gmail.com>

Switching SFX → Speech/Music left params.video_guide in the shared form. generation.music then rejected the leftover guide as an active residual, and generation.speech rejected the URL against its empty-only catalog enum. Stash the guide with the SFX tab, treat a blank residual as inactive, and do not force '' onto audio Load Settings after the tab-owned restore. Co-authored-by: ignaciodelcano+dcl <ignaciodelcano+dcl@gmail.com>

SFX escribe MMAudio_prompt y sfx_text_weight en el formulario compartido. Al pasar a Speech o Music, generation.speech rechazaba el prompt activo y generation.music rechazaba el peso desconocido. El adaptador del formulario los descarta; el envelope cerrado de MCP sigue fallando en cerrado. Co-authored-by: ignaciodelcano+dcl <ignaciodelcano+dcl@gmail.com>

# Conflicts: # ui/tests/studioAudioReferenceState.test.ts

cursor · 2026-09-09T21:30:10Z

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_98557ac1-cc70-472b-baa1-0f0975fdfbf7)

IAnMove added 23 commits September 9, 2026 15:10

Constrain speech prompts and split validation helpers

41494d4

feat(studio): share durable speech commands across UI Wizard and MCP

11e26a0

Merge commit '57c980caf2e270009feaf18ee599780d8db27a9d' into feat/sha…

1efe288

…red-audio-tool-commands # Conflicts: # app/services/image_generation_commands.py

Merge commit 'b0fc2dd3525229dfff1fba5d3465053df0454fd5' into feat/sha…

4d5c2ac

…red-audio-tool-commands

feat: add shared tools upscale command contract

8889286

Merge reviewed image recovery folder fix into speech candidate

40365d5

feat(tools): admit upscale through canonical commands and native worker

af2ce04

Merge reviewed recovery and speech candidate into Tools commands

4b80875

# Conflicts: # app/services/image_generation_commands.py

refactor(tools): separate source projection from source validation

ba974fb

fix(tools): identify durable upscale tasks in Activity

9a157fe

fix(tools): keep inactive processor defaults out of native requests

1868f10

docs: explain Wizard and external MCP command recovery

98a585c

fix(tools): scope upscale assets by source workspace

0ce4985

fix(tools): avoid global asset discovery for scoped file URLs

2615bb7

feat(tools): add durable upscale UI command

1651fc6

feat(commands): add local studio music contract

3e2ac38

fix(tools): guard workspace and seed snapshots

aa42e9b

feat(music): admit local Studio music through shared native commands

7b348d7

fix(tools): route quick upscale through durable panel

d50cb1e

Merge remote-tracking branch 'origin/development' into feat/shared-to…

3df3d81

…ol-commands

Merge commit '7b348d78' into feat/shared-tool-commands

ccb9b5e

fix(commands): retain all native operations after integrating music

d62cd30

feat(ui): add Studio music command client

d8fbec2

IAnMove added 5 commits September 9, 2026 17:24

fix(music): keep catalog input fields in sync

667e42c

feat(studio): present and recover native music requests

f308df2

Merge commit 'd62cd30e' into feat/shared-music-commands

87d3343

# Conflicts: # ui/src/i18n/locales/en/studio.json # ui/src/i18n/locales/es/studio.json

test(e2e): cover Tools upscale command gateway

e1ba86c

test(music): cover Studio ACK and store defaults

91b952a

IAnMove added 2 commits September 9, 2026 20:29

refactor(sfx): separate admission identity, guide timing and content …

81a399d

…checks

docs(qa): classify SFX facade test with the canonical inventory reason

74d29e4

IAnMove changed the title ~~feat(studio): share native speech, music and upscale commands~~ feat(studio): share native audio and upscale commands Sep 9, 2026

IAnMove and others added 19 commits September 9, 2026 20:55

fix(sfx): discard stale video options when entering native audio

0220c80

Merge reviewed Music caption fix from PR #284

91341e5

fix(studio): keep model-specific captions and report the admitted media

5148f1e

Merge SFX pack intent fix from PR #285 for recovery hardening

9f78d1d

fix(sfx): retain per-clip receipts and partial pack recovery

756118c

fix(wizard): retain explicit model and negative prompt in SFX packs

2b8a81a

fix(wizard): preserve authored SFX packs and block video fallback

94ab006

fix(studio): isolate audio references across submode changes

747875a

fix(studio): align audio tabs and show selected voice references

1b9fcdd

fix(studio): keep audio reference selectors after late model defaults

5cb8107

Merge reviewed SFX form residue fix from PR #286

aeea671

fix(studio): restore saved SFX controls and preserve audio duration

a3e3771

fix(studio): report submission truthfully and guard repeat clicks

094ef9d

fix(studio): pin gallery restore and reroll to the clicked output

af5362b

refactor(studio): share typed submission failure handling

07be48c

cursor Bot mentioned this pull request Sep 9, 2026

fix(studio): no enviar residuos de SFX en generation.speech y generation.music #288

Merged

25 tasks

IAnMove added 4 commits September 9, 2026 23:03

Merge reviewed SFX video-guide tab isolation from PR #287

091c0ec

# Conflicts: # ui/tests/studioAudioReferenceState.test.ts

Merge reviewed SFX form isolation from PR #288

f146357

Share audio form admission guards and project SFX-owned text once

3c9b7fe

Drop music-only form controls when preparing Speech

b18b444

IAnMove marked this pull request as ready for review September 9, 2026 21:29

IAnMove merged commit 06f8ea2 into development Sep 9, 2026
13 checks passed

Navigation Menu

feat(studio): share native audio and upscale commands - #280

feat(studio): share native audio and upscale commands#280
IAnMove merged 76 commits into
developmentfrom
feat/shared-tool-commands

IAnMove commented Sep 9, 2026 •

edited by cursor Bot

Loading

Uh oh!

github-actions Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 9, 2026 •

edited

Loading

Uh oh!

Uh oh!

cursor Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

IAnMove commented Sep 9, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes

Validation

Remaining work

Uh oh!

github-actions Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review — Loreframe Studio

Findings

Changed files

CONTRIBUTING checklist

Uh oh!

github-actions Bot commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code health

Quality score: 58.6/100

Most complex functions

Trend vs baseline

Warnings

Uh oh!

Uh oh!

cursor Bot commented Sep 9, 2026

Bugbot couldn't run - usage limit reached

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

IAnMove commented Sep 9, 2026 •

edited by cursor Bot

Loading

github-actions Bot commented Sep 9, 2026 •

edited

Loading

github-actions Bot commented Sep 9, 2026 •

edited

Loading